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ABSTRACT 


In Command and Control, decisions require the fusion of 
inputs from a number of subordinate decision-makers during the 
situation assessment process. The tactical operating 
environment often introduces stress into the team's decision¬ 
making process. The Office of Naval Technology in Arlington, 
VA has sponsored research under the Tactical Decision-Making 
Under Stress (TADMUS) program to study ways to minimize the 
degradation to the teams' effectiveness during these periods. 
Under the TADMUS Project, the Tactical Adaptation and 
Coordination Training (TACT) experiment was designed by 
Alphatech, INC. to test theories on individual and team 
training techniques that were hypothesized to mitigate the 
effects of stress during tactical situations of interest (high 
and low stress scenarios). 

In a detailed review of the data gathered during the 
experiment, it is concluded that the training strategies were 
indeed effective in significantly altering the subject teams 
ability to perform under the test conditions. Additionally, 
there were no conclusive findings that the level of stress, as 
presented in the experiment, had a significant effect on the 
performance of the teams. 
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INTRODUCTION 


A. BACKGROUND 

In the Navy/Marine Corps White Paper ...From the Sea, a 
fundamental change in strategy away from a focus on a global 
threat to a focus on regional challenges and opportunities is 
presented by Navy leadership as a result of a fundamental 
shift in the national security policy. As Joint Task Force 
members. Naval Forces will be full participants in the 
principal elements of the National Security Strategy — 
strategic deterrence and defense, forward presence, crisis 
response, and reconstitution. ■ Maritime forces are 
particularly well suited for the forward presence and crisis 
response missions which would likely be conducted in the 
complex operating environment of the "littoral" or near the 
coastlines of the world. In shifting away from open-ocean 
warfighting on the sea, the leadership has recognized that if 
forces are to be successful they must be trained and equipped 
to make near instantaneous correct decisions. Commanders will 
be making those decisions while operating task forces 
constrained by less room to maneuver and a far shorter 
reaction time to a wider range of more lethal threats. 

In littoral regions, friends, adversaries, and neutrals 
are all operating within confined and congested water and 
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airspace. The normal density of contacts encountered while 
operating near land makes identification and situation 
assessment extremely difficult for the forces. (O'Keefe, 
Kelso, and Mundy, 1992) As a result of the probable need to 
deploy forces into difficult operating environments, where 
mistakes can be catastrophic, the Navy has pursued programs 
designed to improve tactical decision-making and performance 
in stressful situations. 

The Tactical Decision-Making Under Stress (TADMUS) program 
was initiated in an effort to better understand how tactical 
decisions are made during high stress periods, when commanders 
often are faced with grave consequences as a result of 
incorrect decisions, which are often made based upon ambiguous 
data and uncertain information. Past programs concentrated on 
providing more time for decision making by speeding up the 
processing, dissemination and displaying of information. 
(Smith and Grossman, 1993) The TADMUS program's primary 
objective is to apply recent developments in decision theory, 
individual and team training, and information display to the 
problem of enhancing tactical decision quality under 
conditions of stress. The following five tasks comprise the 
program's objectives: 

1. Definition and measurement of critical decision tasks. 

2. Examination of stress effects on decision-making. 

3. Development of decision support principles. 

4. Development of training and simulation principles. 
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5. Development of display principles. 

The program consists of two projects; both of which are 
sponsored by the Office of Naval Technology, which is located 
in Arlington, VA. Project RM33T40 is principally concerned 
with the development of decision support principles and 
display principles for decision support systems. Project 
RM33T60 is principally concerned with development of training 
and simulation principles to counteract stress. This thesis 
reports on the Tactical Adaptation and Coordination Training 
(TACT) experiment which was designed to test teamwork, 
coordination, and adaptation training strategies in support of 
completing task objective four. (Malecki and Collyer, 1992) 

B. PURPOSE 

1. Tactical Decision-Making In A Combat Environment 

In recent history, catastrophic incidents such as the 
one involving the USS Stark and later the USS Vincennes have 
focused attention on the human factor in decision-making under 
stressful conditions. In retrospect, it is now acknowledged 
that improved training and support must be provided to the 
decision-maker to aid him in sorting through confusing 
situations where it is often not clear who the enemy is or 
what his intentions may be. (Malecki and Collyer, 1990) The 
TADMUS project was initiated to explore methodologies to 
improve the decision-making process in teams under stress. 
The TACT experiment is one in a continuing series of TADMUS 
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program experiments. TACT was specifically designed to test 
theories on individual and team training techniques that were 
hypothesized to increase team performance during periods of 
high stress. Using the Decision-Making Evaluation Facility 
for Tactical Teams (DEFTT), information processing similar to 
that of an actual AEGIS cruiser combat information center 
(CIC) environment was simulated. Based upon variations in 
levels of training and differing levels of stressful 
scenarios, the DEFTT lab was used to measure the effect of the 
training techniques in question and their ability to improve 
team performance versus selected measures of performance. 
(Malecki and Collyer, 1992) 

2. Questions 

The TACT experiment was designed to test the efficacy 
of training techniques aimed at enhancing teamwork skills and 
providing team coordination and adaptation strategies. The 
coordination and adaptation strategies would be tested to see 
if they helped the teams maintain their level of performance 
in the face of stress. The particular research questions are 
as follows: 

1. If team members are taught to monitor other team members 
and offer assistance during periods of high stress 
(presence of heavy workload on a team member) will the 
teams' overall level of performance improve? 

2. If teams are given a technique to achieve a shared 
mental model of the current tactical picture, such as 
structured TAO situation assessments, would their 
performance level be improved? 
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Additional questions of interest for this thesis are: 

1. Does an analysis of the experiment provide insight into 
the decision making process that can be used to improve the 
effectiveness of the training? 

2. What can be done to improve the TACT experiment? 

3. Approach 

For the TACT experiment, the overall approach was to 
identify appropriate candidate adaptation and coordination 
strategies; test those strategies in a laboratory setting; and 
then evaluate the effectiveness of those strategies. After 
two hours of initial DEFTT lab orientation training, which 
included individual workstation familiarization, the teams 
engaged in two practice scenarios where they were allowed to 
interrupt and ask questions. Prior to running each new 
scenario, there was a scenario prebrief which included 
specific tasking and a current situational update for the 
upcoming scenario. Upon completion of the scenarios, test 
subjects were given questionnaires which were used to gather 
data on their reactions to the scenarios in terms of the 
level (s) of stress that were present and their perceived task 
workload. Once the familiarization runs were completed, each 
team participated in two combat scenarios, one low stress and 
one high stress. Observations of the teams' performance were 
recorded using audio, video, and written observer rating 
forms. Training intervention was then conducted, immediately 
followed by two combat scenarios. The data obtained in the 
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later two simulations was compared to data obtained in the 
first two simulations to determine the effectiveness of the 
training. Minitab, a computer based statistical package, was 
later used as the prime analysis tool in an attempt to measure 
significant differences in the levels of observed performance 
of the teams as a result of receiving intervention training.' 

4. Anticipated Results 

It was hypothesized that the coordination and 

adaptation training strategies that were tested in the TACT 

experiment would be effective in counteracting the negative 

effects of stress and would thus improve team performance 

during periods of high stress. The overall premise was 

well trained teams cope with stress through internal 
mechanisms of decision strategy adaptation, coordination 
strategy adaptation, and structural reconfiguration, in an 
effort to keep performance at a required level while 
maintaining stress below an acceptable threshold (Serfaty, 
Entin, Deckert, and Volpe, 1993). 

The experiment was designed to look at the following specific 

points and their relationship to managing the effects of 

stress: 

1. Training for implicit coordination. 

2. Training for team adaptation to stress. 

3. Periodic sharing of the TAO's assessment of the 

situation. 

4. Enhancement of the team's structural flexibility. 


1 Minitab was the computer-based statistical analysis software 
package of choice used by the author. 
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If the experiment is successful, then it will have 
demonstrated that teams in a laboratory setting can be trained 
to recognize the presence of stress, reconfigure their 
structure(s) and adjust their strategies in order to maintain 
an acceptable level of performance when operating in a 
tactically challenging environment. 

a. Support for anticipated results 

Review of the literature suggests that teams use 
explicit coordination during periods of low stress and 
implicit coordination during periods of increasing stress. 
Explicit coordination requires more communications as team 
members generally respond to requests for information etc. 
from others as they coordinate their efforts. During periods 
of stress, implicit coordination becomes the desired norm. 
This strategy requires team members to become familiar with 
others needs; to be able to anticipate requests and respond or 
provide support without prompting. Implicit coordination 
relies upon team members having a shared mental model of the 
situation. 2 The team members are then able to anticipate the 
needs and desires of others that are related to managing the 
crucial task(s) of the moment and they will thus refrain from 
introducing noise to the situation assessment and decision¬ 
making process. (Serfaty, Entin, Deckert and Volpe, 1993) 

2 A detailed discussion of the importance of teams having a 
common shared mental model is presented in Rouse, Cannon-Bowers and 
Salas (1992). 
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During the TACT experiment, the TAO is prompted to 
periodically share his assessment of the tactical picture in 
order to provide the appropriate mental model which is 
required for implicit coordination. After the TAO provides 
his assessment, members then only need to communicate relevant 
information that deviates from his assessment. 

The literature also suggests that teams should 
maintain the flexibility to restructure when faced with 
stress. The hypothesis is that with effective cross training 
and functional task familiarization, superior teams will be 
able to reconfigure their structure to support members who are 
experiencing task overload at their workstations. This 
strategy should enable the team to regulate the workload 
imposed by the external stressors. (Serfaty, Entin, Deckert 
and Volpe, 1993) During the TACT experiment, use of this 
technique was encouraged and thus subsequently observed being 
used. 

C. EXPERIMENTAL PARTICIPANTS 

The TACT experiment was conducted at two sites. Students 
from the Joint Command Control and Communications (JC 3 ) 
curriculum at the Naval Postgraduate School (NPS) in Monterey, 
CA and personnel from the Surface Warfare Officers' School 
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The 


Command in Newport, RI participated as test subjects. 3 
experiment was run by contract personnel from Alphatech, INC. 
and representatives from the Naval Training Systems Center, 
Orlando, FL. At NPS, student participants were divided into 
six teams with five members each. Two teams acted as the 
control group, two teams received TACT training and the 
remaining two teams received, TACT and information structure 
training. Each subject group had five operator positions, 
tactical action officer (TAO), electronic warfare supervisor 
(EWS), anti-air warfare coordinator (AAWC), tactical 
information coordinator (TIC) and identification supervisor 
(IDS). The slate of operators reflected the current suite of 
key players in a typical Navy Aegis Combat Information Center 
(CIC). In addition to serving as test subjects, additional 
students assisted contractor personnel in the mechanics of 
running the experiment such as providing role play and 
completing subjective evaluations of team performance. At NPS 
a third group was responsible for reporting their 
interpretation of the data gathered during the experiment as 
a part of their study of the JC 3 curriculum. 


3 This thesis documents those activities observed at NPS 
Monterey, CA where the author was an experimental participant and 
student in the JC 3 curriculum. 
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D. EXPERIMENT SCOPE 


The DEFTT laboratory was used to conduct four simulations 
(two of high stress scenarios, two of low stress scenarios) to 
evaluate the effects of the adaptation and coordination 
training strategies. If the strategies prove effective in the 
laboratory tests, then those promising techniques would be 
further refined and subjected to additional testing in at-sea 
experiments. The effective strategies will eventually be 
developed into a series of recommended principles, guidelines, 
and methodologies for training tactical decision-making teams 
throughout the fleet. (Malecki and Collyer, 1990) 
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II. EXPERIMENTAL DESIGN 


A. OVERVIEW 

The TACT experiment was developed by Alphatech, INC. to 
test hypotheses regarding training interventions that would 
reduce the impact of stress on tactical decision-making. This 
four stage experiment used the DEFTT laboratory to provide 
realistic simulations of AAW scenarios. (Entin, 1994) In 
order to keep the scope of the experiment to a manageable 
size, six key positions were identified as those most critical 
to replicating the AAW decision-making process inside of a 
ship's CIC. The six positions which are supported by the 
laboratory are: commanding officer (CO), TAO, AAWC, TIC, IDS, 
and EWS. 4 The lab also provides all of the necessary tactical 
cues required to support situation assessment and decision¬ 
making while it records the actions of team members for later 
analysis as they play through the presented scenarios. 
(Malecki and Collyer, 1992) Further details describing the 
setup, the hypotheses, assumptions, statistical design, 
measures and instrumentation are provided in the following 
sections. 


4 The position of Commanding Officer was not utilized during 
the TACT experiment. 
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B. SETUP 


The setup is comprised of the following four general 
categories: physical, test subjects, special equipment, 

schedule of trials. 

1. Physical 

The DEFTT lab was designed to provide good functional 
realism of an operational AAW environment. The setup consists 
of six IBM-AT 386 personal computers with Aegis display 
system, command and display system, and electronic warfare 
supervisor software modules. The six personal computers, 
which serve as operator workstations, are networked to a 
Hewlett-Packard 9000/345 experimental control station (ECS). 
The ECS generates and controls experimental scenarios, 
supports a multi-channel communications system, and runs a 
Barco Graphics Large Screen Display. 5 An ideal (uncluttered) 
link picture with Naval Tactical Data System (NTDS) symbology 
is available to all of the operators via their personal 
computer workstations. (Malecki and Collyer, 1992) 
a. Scenarios 

The DEFTT scenarios are set in the Arabian Gulf 
area of operations. This geographic region is a particularly 
good setting for potential "real world" incidents due to its 
volatility and high density of contacts. The scenarios are 


5 The system is capable of time-stamped recording of all 
verbal communications among team members. 
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filled with ambiguous situations which are often left 
unresolved for the team players. The experiment was designed 
to prohibit decisions that were made in a prior scenario from 
influencing decisions made in later scenarios. This design 
eliminated a need to provide feedback to teams on the 
consequences of their actions and the actions of any contacts 
of interest that were perceived as threats to the team. This 
technique was seen as increasing the realism of the scenarios 
since many real world encounters end with questions of true 
intent left unresolved. 6 (Hutchins and Kowalski, 1993) 
b . Task Demands 

The scenarios were developed to provide a stressful 
environment for teams to be observed working within. 7 They 
were designed to require completion of many complex tasks 
within short periods of time. Contacts of interest were 
blended in with neutral/unknown and friendly tracks, thus 
making the determination of their identity and intent a more 
complicated process. The uncertainty and ambiguity makes the 
required application of the rules of engagement (ROE) more 
difficult for the teams. Even in cases where engagement 


6 The primary purpose of the experiment is not to judge teams 
regarding their right or wrong decisions, but rather to examine 
their ability to adapt and coordinate their actions when presented 
with ambiguous threatening scenarios. 

7 For experimental purposes, the level of stress was 
manipulated through the use of two variables: environmental 
ambiguity and operator workload. (Entin, 1994) 
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criteria are met, per the correct application of the ROE, the 
more appropriate decision might be not to engage. 
Additionally, the workload on team members is increased by 
both internal and external communications requirements, while 
very little support from off-ship assets is provided to aid in 
clarifying any ambiguous situations. (Hutchins and Duffy, 
1993) 

2. Test Subjects 

The six test subject teams were composed of five 
members each. The team members were tasked with filling five 
of the six key decision-making positions found in a typical 
Aegis CIC. All of the subjects were student members of the 
JC 3 curriculum at NPS. The subjects represented all the 
services and included one civilian student from the National 
Security Agency (NSA). All teams had members with varying 
levels of military experience, but all of the appointed TAOs 
had significant Navy CIC operational experience. All of the 
officers were of the 0-3 and 0-4 paygrades. The subjects were 
unaware of which type of training they had received. 
Additionally, they were instructed not to discuss the 
experiment with the other teams until all teams had completed 
the experiment. This was done to prevent any potential 
compromise of the experiment. 
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3. Special Equipment 

The primary equipment required for conducting the 
experiment was described in the physical setup. A VHS 
recorder was the only additional piece of special equipment 
that was not previously mentioned. The videotape recorder was 
used to support later analysis by recording scenario events 
from the TAO's large screen display and the associated 
communications (audio) that was occurring at the time amongst 
the team members. 

4. Schedule 

The experiment was divided into four phases and was 
conducted in the following sequence; 

1. Stage 0 — basic instruction and training for all teams. 

2. Stage 1 — pre-intervention data collection. 

3. Stage 2 — intervention training. 

4. Stage 3 — post-intervention data collection. 
Intervention training, stage 2, consisted of one of the 
following three training levels: 

1. CONTROL—No significant training or other pertinent 
information was provided. This served as a baseline for 
team comparison. 

2. TACT—Subjects were taught to identify signs and 
symptoms of stress. Subjects were then taught several team 
strategies to adapt to stress with a focus on team 
coordination. Scripted video presentations, which 
demonstrated good and bad application of the team behavior 
strategies, were then viewed. Subjects were then allowed 
to practice what they had learned on the DEFTT simulator 
using two practice scenarios. To complete the intervention 
training, subjects viewed a summary videotape on the 
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importance/principles of teamwork presented by a retired 
Rear Admiral. 

3. TACT and Information Structure (TACT plus)—In addition 
to the TACT training, the TAO was prompted every three 
minutes to provide a situation report (sitrep) over the 
network so that other workstations would understand his 
assessment of the overall tactical picture. Additionally, 
team members were instructed on how to interpret and best 
utilize the sitrep information. 

At stage 0 (prior to commencing the experiment), each of the 

teams were provided two hours of training. This training 

consisted of task familiarization, two slide briefs, and a 30 

minute proficiency run using the simulator. At stages 1 and 

3, each teams' performance was observed and evaluated during 

both a high stress and a low stress scenario. Stage 1 data 

observations were then compared to stage 3 data observations 

for indications of the effectiveness of the training conducted 

during stage 2. (Entin, 1994) 

C. HYPOTHESES 

The primary purpose of the TACT experiment was to test the 
hypothesis that training can mitigate the negative effects of 
stress on team performance. The experiment specifically 
focuses on looking at coordination and adaptation strategies 
and whether or not they have the desired effect which is 
suggested by the theories set forth in Chapter I. The 
contractor designed the experiment to look at several 
different measures; anyone of which could be used to support 
accepting the premise that their proposed training strategies 
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did indeed effect team performance during periods of stress. 
(Entin, 1994) This thesis will limit its focus to only two of 
the five measures that are supported by the experiment and 
used by the contractor. 8 The hypotheses of interest are: 

H ol : Training will not effect overall team performance. 

H a |J Training will effect overall team performance. 

H o2 : Training will not effect teamwork. 

H a2 : Training will effect teamwork. 

D. ASSUMPTIONS 

There were four predominate assumptions associated with 
the TACT experiment. First, DEFTT is a legitimate simulation 
of an actual Aegis CIC environment (Entin,1994). 9 It is 
therefore a valid tool for testing the experimental 
condition(s) of interest (decision-making in an at-sea 
tactical environment during high and low stress periods). 
Another assumption was that after initial familiarization 
training, all teams were near the same level of competence and 
understanding of the functionality/buttonology of their 
respective watchstations. Therefore, once the testing began 
any learning curve effect would be negligible. Thirdly, the 
observers' ratings of team performance were quantitatively 

8 A more complete listing of the measures will be provided 
later in the paper. 

9 Experts have accepted DEFTT as a valid simulation of the 
higher level functional decision making process that occurs within 
an Aegis AAW environment. 
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consistent throughout the course of the experiment. The final 
significant assumption was that the subjects were willing and 
enthusiastic participants. This assumption was necessary 
since the subjects were not volunteers. 10 

E. STATISTICAL DESIGN OF EXPERIMENT 

The statistical design of this experiment was modeled 
after Campbell and Stanley's Pretest-Posttest Control Group 
Design 4 (Entin, 1994). Design 4 (adapted by Alphatech, INC. 
for the TACT experiment) uses random assignment of training 
techniques to each team, in that the team conditions (i.e., 
control, tact, or tact plus) and when they would be exposed to 
differing experimental variables (i.e., low stress or high 
stress scenario) were assigned prior to the subjects being 
assigned to the teams. This randomization would likely 
further negate a potential compromise of the experiment that 
might take place if team members were to prematurely discuss 
their roles and the training scenarios which they had 
participated in. This experimental design also supports 
multivariate analysis which is capable of looking at more 
than one independent and/or more than one dependent variable. 
(Campbell and Stanley, 1965) The experimental schedule, is 
presented in the following table. 


10 The author's personal observation was one of enthusiastic 
subjects who appeared to put forth their best efforts commensurate 
with their abilities. 
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Table I: EXPERIMENTAL SCHEDULE 


CONDITION 

TEAM ID # 

PRE-TRAINING 

POST¬ 

TRAINING 


N5 

2- 

2+ 



1+ 

1- 

CONTROL 

N2 

1+ 

1- 



2- 

2 + 


N3 

1+ 

1- 



2- 

2 + 

TACT 

N1 

1- 

1+ 



2+ 

2- 


N6 

2- 

2+ 



1+ 

1- 

TACT & 
STRUCTURE 

N4 

1+ 

1- 



2- 

2+ 


Table I depicts simulation assignments to the six teams 
involved in the experiment. A key for the table is 
provided below. 


"1—" => 

scenario 

one, 

low 

stress 

"1+" => 

scenario 

one, 

high 

stress 

"2-" => 

scenario 

two, 

low 

stress 

"2+" => 

scenario 

two, 

high 

stress 


F. MEASURES 

The TACT experiment used four methods of measuring team 
performance. The measurement instruments consisted of pre and 
post-mission questionnaires, the NASA Task Load Index (TLX), 
trained observers conducting performance assessments, and 
observers quantitatively measuring verbal communication rates. 
(Entin, 1994) Only that data collected as subjective measures 
of teamwork and team performance will be analyzed within this 
paper. 
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1. Pre-Mission Questionnaires 

The pre-mission questionnaires were administered to 
subjects before and after intervention (stage 2) training. 
The questions were designed to assess the perceived congruence 
among team members of the same mental model of the tactical 
situation. (Entin, 1994) If the training was successful, 
then the after intervention training questionnaires should 
indicate an increased perception from members that they and 
the team are more likely to share the same mental picture of 
the situation as a result of applying the techniques suggested 
during the stage 2 training. 

2. Post-Mission Questionnaires 

The post-mission questionnaires were administered to 
subjects after each trial. The questions were designed to 
assess the level of confidence within the team, the amount of 
cross-monitoring of teammates workload (increased awareness of 
the presence of stress) and whether the level of assistance 
and anticipation of other members needs had increased. When 
combined with the pre-mission questionnaires, the post-mission 
questionnaires formed the basis for a subjective self 
assessment of team performance. (Entin, 1994) 

3. Subjective Workload 

The NASA TLX was used to determine the workload that 
subjects felt they were exposed to during the course of the 
experiment. This rating procedure relates workload to 
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demands imposed on the subject and to the interaction of the 
subject with the task. The TLX responses are combined to 
produce overall weighted workload scores. (Hutchins and 
Kowalski, 1993) This measure could then be used to look at 
perceived subject workload as it relates to stress level and 
if the member's perceived workload was reduced as a result of 
being exposed to stage 2 training strategies. The implication 
would be that with a reduction in the perceived workload, as 
a result of the intervention training, an acceptable level of 
performance will be maintained. 

4. Teamwork and Performance 

The subject teams were rated by two trained observers. 
The observers graded teamwork on the basis of 15 individual 
items which were deemed to be appropriate indicators of 
teamwork. The combined items are referred to as the AAW Team 
Observation Measure (ATOM). 11 In addition to the teamwork 
ratings, the observers were also evaluating the teams' 


11 ATOM is a team process measure. It was developed based upon 
a critical incident approach and refined during workshops. 
Dimensions of the AAW team process include: communication, team 
orientation, team leadership, monitoring, backup, feedback, and 
coordination. These behaviors are scored on a scale from 1 to 7 
for each event in the scenario. Note: the team leadership 
dimension was not evaluated during the TACT experiment. (Malecki 
and Collyer, 1992) 
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performance on the basis of the 12 individual items which are 
referred to as the AAW Team Performance Index (ATPI). 12 

5. Team Communication and Coordination 

Observers used matrices to tally communications rates 
during the scenarios. These quantitative measures were to be 
used to assess any reduction in the requirement for 
information exchange as a result of the training intervention. 
(Entin, 1994) In theory, members should now be more apt to 
anticipate communications needs and therefore pass pertinent 
information before it is requested. 

G. INSTRUMENTATION 

Samples of the various measuring instruments are provided 
in Appendix A. 

H. TESTING AND PILOT TRIALS 

A series of preliminary tests were conducted prior to the 
TACT experiment. Those tests looked at the performance 
measurement instruments, the stress evaluation methodology, 
and the performance of the DEFTT laboratory itself. An 
initial test of the performance measures, using instructors at 
the Aegis Training Unit, Moorestown, NJ, revealed that the 
scales were useable and that they appeared to be able to 


12 ATPI is a team outcome measure. It is scored on a scale 
from 1 to 7, and is based on the frequency of effective and 
ineffective behaviors exhibited by the AAW team. ATPI has 
demonstrated a sensitivity for detecting performance differences 
due to the presence of stress. (Malecki and Collyer, 1992) 
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distinguish between levels of performance. A preliminary 
stress evaluation methodology was developed, which uses a 
stress matrix, to evaluate stress levels within the baseline 
scenarios. The matrix became a tool to measure the number of 
targets (workload) versus the information available about the 
target (ambiguity level). Other testing and pilot studies 
were done on the DEFTT lab to ensure that it is capable of: 
recording time-stamped multiplexed channel data; supporting 
workstation simulation at both operator and command levels; 
and supporting networked simulation for simultaneously testing 
of multi-operator teams. In addition, the DEFTT scenarios 
were reviewed by experts to evaluate their "stressfulness" and 
to develop relevant background materials (i.e., geopolitical, 
ROE, order of battle) to be used during the experiment. 
(Malecki and Collyer, 1992) 

Other experiments, which have preceded TACT, provided key 
findings which were further examined during the TACT 
experiment. Among those prior experiments were the SAINT 
experiment and the CHIPS experiment. The SAINT experiment 
studied the effects of team leader feedback on the situation 
assessment process in AAW teams (Gough, 1992), while CHIPS 
attempted to assess the impact of human cognitive limitations 
on team performance (Armbruster, 1993) . The preceding TADMUS 
program experiments yielded results which supported TACT 
hypotheses and/or required further evaluation during the TACT 
experiment. 


23 




III. DATA DESCRIPTION 


The TACT experiment collected both quantitative and 
qualitative data for analysis. Data collection instruments 
included pre and post-mission questionnaires, the NASA TLX 
workload index, observer rating forms, and team communication 
recording forms. The various collection instruments are 
included in Appendix A. Additionally, a videotape recording 
of the proceedings was made to support later analysis. 13 

A. EXAMPLE OF RAW DATA 

Raw qualitative data was recorded by trained observers 
using the Teamwork and Performance: Observer's Rating Form. 
For each observation, an interval marking scale from one to 
seven was provided. The forms provided brief descriptions of 
the behaviors that should be observed for the lowest ratings 
on the scale and brief descriptions of the behaviors that 
should be observed and associated with the highest ratings on 
the scale. Using the descriptors as guides, the observers 
then rated each team's performance in the area of interest. 

The author uses the data obtained from the rating forms to 
support all analysis for this thesis. A brief discussion of 


13 The videotape equipment was used to record the multi-channel 
communications (both internal and external) and the TAO's tactical 
presentation as seen on the command and decision geographic 
situation summary or large screen display. 
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the uses of the pre and post-mission questionnaires, the TLX 
workload index, and the team communication recording forms are 
presented in the following text in order to provide a more 
complete picture of the experiment. 

1. Pre and Post-mission questionnaires—these forms 
provided scales ranging from one to seven for subjects to 
annotate their feelings regarding their confidence in the 
teams' and their own abilities to complete the assigned 
tasking and to ascertain if the subjects felt they were 
able to assist others. These questionnaires were designed 
to support testing of the shared mental model theories 
presented in Chapter I. 

2. TLX workload index—these forms provided a scale for 
test subjects to rate the previously completed missions in 
terms of the following six areas: mental demand, physical 
demand, temporal (time pressure) demand, performance, 
effort, and frustration. 

3. CIC Team Communication Recording Forms—the TAO's 
communications and those of the team members were recorded 
by observers using simple tick marks to note each 
occurrence in a matrix format. The matrix data will 
support analysis of communications rates and the nature of 
the communications in terms of them being requests, 
transfers or acknowledgements. 

All of the data collected and described above will be used by 
Alphatech, INC. in their analysis. Additional data analysis, 
not discussed here, will be conducted by the Naval Training 
Systems Center and by the Naval Command, Control and Ocean 
Surveillance Center (NCCOSC) RDT&E Division as they also 
review the TACT experiment. 


B. DATA PROBLEMS 

No data collection or interpretation problems were 
encountered relevant to this analysis. 
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C. DATA TABLE CODING SCHEME 

Each observer's scores were recorded on the rating forms. 
Those values, plus specific identification and experiment 
control variables were transcribed into a data table of 34 
columns of variables by 48 rows of iterations. (See Appendix 
B.) The first seven columns consist of identification and 
experiment control variables. Of the remaining 27 columns, 
the first 15 correlate to the teamwork measures of the rating 
form. The next eight columns measure team performance in 
general and the final four columns are associated with 
performance specific to the particular scenario. The coding 
scheme for the first seven columns is presented in Table II. 
The remainder of the columnar values are transcribed directly 
from Appendix B. 

D. DATA REDUCTION 

The first step in data reduction was to sort the data 
spreadsheet by groupings which would support analysis of the 
hypotheses in question. After sorting the data, averages were 
computed for performance and teamwork scores using the ATPI 
and ATOM measures discussed in Chapter II. Row averages for 
all of the data entries were then extracted along with their 
associated identifiers. After reducing and arranging the 
initial spreadsheet into a more manageable format, further 
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Table II: DATA CODING SCHEME 


COLUMN 

VARIABLE 

IDENTIFIER 

A 

Site ID 

1 = SWOS Newport; 

2 = NPS Monterey 

B 

Experimental 

Condition 

1 = Control; 

2 = TACT; 

3 - TACT + 

C 

Team ID 

One through six 

D 

Observation 

1 = Pretraining; 

2 = Posttraining 

E 

Scenario # 

One or two 

F 

Stress Level 

1 = Low; 

2 = High 

G 

Trial # 

One through four 


subdivisions of columns were completed as required to support 
a detailed categorical analysis of the experimental data. The 
modified data spreadsheet is provided in Appendix B. 
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IV. 


ANALYSIS 


This chapter describes the analysis procedures used for 
the TACT experiment. It sets forth the analysis plan and then 
provides the detailed steps that were conducted during the 
process. In the final section of the chapter, the detailed 
results of the experiment are presented. 

A. ANALYSIS PLAN 

The analysis of experimental results was conducted in two 
phases. First, the ATPI outcome measures were used to rate 
team performance effectiveness, then the ATOM process measures 
were used to evaluate teamwork behaviors. Using the data 
collected from the observer's rating forms, the experiment 
attempts to detect differences in team performance and 
teamwork evaluation scores where it is expected that some 
experimental treatments will be more effective than others. 
Based on the findings, the two null hypotheses presented in 
Chapter II will be rejected if the sample evidence contradicts 
them and provides strong support for the alternative 
hypotheses. 

The statistical analysis of the test data was completed 
with the aid of Minitab. Minitab generates a p-value that 
indicates the probability of observing an outcome like the one 
actually observed or more unusual, under the assumption that 
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the given dependent variable is effected the same by all 
levels of the independent variables (when the null hypothesis 
is true). Tests of significance at a = 0.05 will be used as 
rejection criteria for the null hypotheses. Using Minitab, 
when the p-value is less than the critical value (a = 0.05) 
one can be 95% certain that any change in the dependent 
variable was caused by a change in the independent variable 
(treatment condition), not a random occurrence. 14 

B. METHODOLOGY 

The recommended analysis for "Design 4" is based on gain 
score, using the analysis of covariance (ANCOVA) tests 
(Campbell and Stanley, 1965). In addition to these tests, 
normality and scatter plots are used to provide additional 
insight during the analysis process. For each hypothesis, an 
initial ANCOVA test is run on the data. The posttraining 
intervention test score is used as the dependent variable and 
the pretraining intervention test score as the covariate. The 
gain scores were then determined by comparing pretraining and 
posttraining scores amongst all test groups. Using the gain 


14 The significance level a represents the probability of 
rejecting the null hypothesis when the null hypothesis is true. 
Using Minitab, if the p-value is less than a the null hypothesis is 
rejected. 
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scores, a second ANCOVA test is then conducted with 
pretraining scores being used as the covariate. 15 

C. RESULTS 

1. Hypothesis One: Test of Performance Measures 

The ANCOVA test indicates that the covariate, 
pretraining score, has a highly significant effect on the post 
performance score (p-value = 0.002). Additionally, the test 
indicates that one of the experimental factors in question 
(level of training) does indeed apparently account for a 
statistical difference in the posttraining performance scores 
of the teams (p-value = 0.011). There is no indication that 
the other independent variable (level of stress) significantly 
effects the performance scores across the experimental 
conditions. The Minitab printout is provided in Table III. 

Graphical representations of the pretraining and 
posttraining mean performance scores were examined for the 
three experimental conditions at the two levels of stress. 
These two bar graphs when examined together show that the 
control groups high and low stress performance was essentially 
unchanged throughout the experiment. At the other two levels 
of treatment, mean performance scores increased for both 
stress conditions. The observer is left to conclude that the 


15 The covariate is an uncontrolled experimental variable that 
influences the response but is itself unaffected by experimental 
factors. Using the ANCOVA technique, adjustments are made for the 
covariates effect on the dependent variable. 
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Table III: ANCOVA FOR POSTTRAINING PERFORMANCE SCORES 


MTB > ANCOVA 'postperf' = cond24 stress24; 
SUBC> Covariates 'preperf'. 

Factor Levels Values 


cond24 3123 

stress24 212 


Analysis of Covariance for postperf 


Source 

DF 

ADJ SS 

MS 

F 

P 

Covariates 

1 

6.5377 

6.5377 

12.83 

0.002 

cond24 

2 

5.8363 

2.9182 

5.73 

0.011 

stress24 

1 

0.3745 

0.3745 

0.74 

0.402 

Error 

19 

9.6808 

0.5095 



Total 

23 

32.2971 




Covariate 

Coeff Stdev 

t-value 


P 

preperf 

0 . 

5436 0.152 

3.582 

0 

.002 


presence of training increases team effectiveness when 
compared to the performance score of the control group. See 
Figure 1 to compare the pre and posttraining scores by the 
three conditions and levels of stress. 

Examination of the scatter plot of the posttraining 
versus the pretraining performance scores confirmed some 
additional expectations. From the ANCOVA test, the positive 
covariate coefficient predicted that low pretraining 
performance scores were an indicator of low posttraining 
performance scores and that a high pretraining score should be 
followed by a high posttraining score. For the most part this 
was the case. The plot also confirmed (as expected) little or 
no movement for about half of the control groups scores while 
dramatic improvement was shown for two of eight of the test 
scenarios where groups received tact only training. The plot 
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Pretraining Performance by Condition and Stress 
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Figure 1: Pretraining and Posttraining Performance Scores 
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also shows that the tact plus groups happened to be strong 
performers prior to receiving the intervention training. See 
Figure 2. 

The ANCOVA test using the performance gain score as 
the dependent variable and the pretraining performance score 
as the covariate also indicates that the level of training is 
responsible for differences in performance scores (p-value = 
0.011). Again stress does not show as a significant factor, 
while the covariate is again determined to be a highly 
significant factor in the analysis. The Minitab printout is 
provided in Table IV. 

Table IV: ANCOVA FOR PERFORMANCE GAIN SCORES 


MTB > ANCOVA 'perfgain' = cond24 stress24; 
SUBC> Covariates 'preperf'. 

Factor Levels Values 


cond24 312 

stress24 212 

Analysis of Covariance for perfgain 


Source 

Covariates 

cond24 

stress24 

Error 

Total 

Covariate 

preperf 


DF ADJ SS 

1 4.6096 

2 5.8363 

1 0.3745 

19 9.6808 

23 17.5931 

Coeff Stdev 

-0.4564 0.152 


MS 
4.6096 
2.9182 
0.3745 
0.5095 

t-value 
-3.008 


F 

9.05 

5.73 

0.74 


P 

0.007 

0.011 

0.402 


P 

0.007 


A plot of the performance gain scores versus 
experimental conditions and stress is also insightful. See 
Figure 3. When the graph is partitioned to show the effects 
of high and low stress, it indicates a relatively dramatic 


33 





{/) 

2 > 

8 

CO 




-r* 

CM 

CO 

T~ 

CM 

CO 

T- 

- 

f- 

CM 

CM 

CM 

O 

<3 

0 

• 


♦ 



to 

IO 


o 

If) 


If) 


p 


to 

CO 


o 

CO 


to 

csi 


o 

csi 


(D 

Cl 

<D 

u 

a. 


o 

CO 

h- 

II 

CO 

u 

CO 

H* 

II 

_C CM 

f e 

II 

CM 

5 

o 


c 

o 

O 

ii 


T« C 

.2 

co S 

CO “O 
CD C 
±3 O 
CO O 


Figure 2: Scatter Plot of Posttraining vs. Pretraining 

Performance Scores 
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Figure 3: Performance Gain Scores by Experimental Conditions 
and Levels of Stress 
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increase in the low stress performance for the tact group and 
a lesser increase in performance under the high stress 
condition. In the tact plus training group, the performance 
gain was high and equal for both low and high stress 
conditions. This is one of the few places in the experiment 
that it appears that stress is a factor. The TAO's situation 
assessments may have been valuable in maintaining the 
performance gain for the tact plus group during the high 
stress scenarios whereas the tact group's performance gain 
under high stress was not as dramatic. It stands to reason 
that if the team is focused on the same contacts of interest 
during a period of relatively high stress then their overall 
performance might be better at least in terms of addressing 
what the decision maker sees as critical. 

The scatter plot of performance gain versus 
pretraining performance score is also presented in Figure 4. 
The plot shows a weak linear relationship for pretraining 
scores and their associated gains. Again the high performers 
had less to gain and the control groups demonstrated little or 
no gain. This is important because it confirms little or no 
performance increase due to any learning curve effect. 

Normality plots for the pretraining, posttraining, and 
gain scores were also included in the analysis (see Appendix 
C) . In all cases, the data points were reasonably close to 
normal. The assumption of normality for the data set was 
required for the above tests to maintain their validity. 
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Scatter Plot of Gain vs. Pretraining Performance 
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2. Hypothesis Two: Test of Teamwork Measures 

The statistical analysis performed for this hypothesis 
is identical to that described for hypothesis one with one 
exception. The parameter being evaluated is teamwork 
behavior. The ANCOVA test indicates that the experimental 
condition (level of training) is significant (p-value = 
0.008). Neither the covariate factor (pretraining teamwork 
score) nor the level of stress are significant. The Minitab 
printout is provided in Table V. 

Graphical representations of the pretraining and 
posttraining mean teamwork behavior scores for the three 
experimental conditions at the two levels of stress are 
presented in Figure 5. These two bar graphs when examined 
together show that the control groups exhibit no evidence of 
improvement in their teamwork behavior score throughout the 
experiment. In contrast, both of the groups that received 


Table V: ANCOVA FOR POSTTRAINING TEAMWORK SCORES 


MTB > ANCOVA 'posttmwk' = cond24 stress24; 


SUBC> Covariates 'pretmwk'. 




Factor Levels 

Values 




cond24 

3 

1 2 

3 



stress24 

2 

1 2 




Analysis of 

Covariance for posttmwk 



Source 

DF 

ADJ SS 

MS 

F 

P 

Covariates 

1 

1.2731 

1.2731 

2.07 

0.166 

cond24 

2 

7.8512 

3.9256 

6.39 

0.008 

stress24 

1 

0.4713 

0.4713 

0.77 

0.392 

Error 

19 

11.6799 

0.6147 



Total 

23 

28.1317 




Covariate 

Coeff Stdev 

t-value 


P 

pretmwk 

0 . 

2605 0.181 

1.439 

0 . 

166 
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Pretraining Teamwork by Condition and Stress 
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Figure 5: Pretraining and Posttraining Teamwork Behavior 

Scores 
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training show positive movement. There is no substantial 
graphical evidence that scores are influenced by the level of 
stress unless one argues that the control group shows a 
substantial decrease in their score when presented with a high 
stress scenario while the other groups perform essentially at 
the same level regardless of the stress factor. 

The scatter plot for the posttraining versus 
pretraining teamwork behavior scores is presented in Figure 6. 
This plot reveals a dispersed grouping of control group scores 
on the low end and a fairly tight cluster of tact plus scores 
on the high end of the scale. The graph also shows a fair 
linear relationship of increasing scores as the level of 
training increases for each of the test conditions. 

The ANCOVA test using the teamwork gain score as the 
dependent variable and the pretraining teamwork score as the 
covariate also indicates that the level of training is 
responsible for differences in teamwork behavior scores (p- 
value = 0.008). Again stress does not appear to be a 
significant factor, while the covariate (p-value = 0.001) is 
a factor. The Minitab printout is provided in Table VI. 

The bar graph representation of the teamwork gain 
scores confirms that the training did have a relatively 
significant effect versus the control group scores. However, 
across the levels of stress, there appears to be no difference 
in the relative gains. The bar graph is presented in Figure 
7. 
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Table VI: ANCOVA FOR TEAMWORK GAIN SCORES 


MTB > ANCOVA 'tmwkgain' = cond24 stress24; 
SUBC> Covariates 'pretmwk'. 

Factor Levels Values 


cond24 3123 

stress24 212 

Analysis of Covariance for tmwkgain 


Source 

DF 

ADJ SS 

MS 

F 

P 

Covariates 

1 

10.2566 

10.2566 

16.68 

0.001 

cond24 

2 

7.8512 

3.9256 

6.39 

0.008 

stress24 

1 

0.4713 

0.4713 

0.77 

0.392 

Error 

19 

11.6799 

0.6147 



Total 

23 

24.5014 





In the next graphical analysis, a scatter plot of gain 
versus pretraining teamwork behavior score is presented in 
Figure 8. The plot shows a negative linear relationship for 
pretraining scores and their associated gains. Again the high 
pretraining performance means less room for gain. As with the 
performance measures, normality plots for the pretraining, 
posttraining, and gain scores were conducted. The data points 
were determined to be reasonably close to normal. As with the 
performance measures of the first hypothesis, the assumption 
of normality for the data was required for the ANCOVA tests to 
maintain their validity. The normality plots are provided in 
Appendix C. 

3. An Additional Observation 

The previous data analysis focused on performance and 
teamwork observation scores as dependent variables with 
experimental condition and level of stress being the 
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Figure 6: Scatter Plot of Posttraining vs. Pretraining 

Teamwork Behavior Scores 
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Figure 7: Teamwork Behavior Gain Scores by Experimental 

Conditions and Levels of Stress 
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Figure 8: Scatter Plot of Gain vs. Pretraining Teamwork 

Behavior Scores 
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independent variables. During the course of the analysis, the 
data was also checked for differences in team scores between 
the two sites. If significant differences in site scores were 
noted, then concerns would be raised regarding the validity of 
the experimental results due to possible biases being 
introduced as a result of two different sets of observers 
being used between the two test sites. 16 

The following group of illustrations (Figures 9-14) 
graphically show the dispersions in scoring patterns between 
the sites. In scatter plots, the NPS subjects tended to be 
homogeneously dispersed, while the SWOS subjects were 
interspersed toward either end of the scoring spectrum 
(control and tact groups rated near the low end of the scale; 
tact plus groups rated near the high end of the scale both 
before and after training). Though the scoring between the 
sites was noted as peculiar, it was felt that the prior 
analysis was not compromised. 17 When viewed separately, the 
posttraining scoring from SWOS seems to show more dramatic 
effects of the training intervention, across the levels. On 


16 To preclude against biased observations, it is suggested 
that the same observers be used for all evaluations and that they 
be kept ignorant as to which test subjects received which 
treatments (Campbell and Stanley, 1965). 

17 In reviewing the data, it was noted that quantitatively 
higher mean scores were given by the NPS observers. However, in 
focusing on the effectiveness of the intervention training 
techniques, it was judged that if all scores were averaged together 
then the effects of the quantitative differences would be negated. 
(Entin, 1994) 
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the other hand, the scoring at NPS showed more realistic near 
equal evaluations of team abilities prior to the teams being 
exposed to training. When the pretraining scores are also 
considered this difference is mostly mitigated. NPS data 
alone also showed improvements with increasing intervention, 
though less dramatic. There is no obvious explanation for 
this apparent anomaly in data between the two sites. 18 


18 As stated in Chapter I, this paper focuses on the experiment 
as observed from NPS. However, for an increased sample size, the 
author uses the data obtained from both experimental test sites 
during the analysis phase. 
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Posttraining Performance by Condition and Site 


& 

8 5 J 

CL 


.4 - 


CO 


4.89583 


4.53125 





■ 

1 

1 

m 

1 

2 

b 

1 

3 

B9 

2 

1 

5 

2 

2 

CD 

2 

3 


Site: 1=SWOS2=NPS ExpCOnd 

Condition: 1=Control 2=Tact 3=Tact+ 

Figure 9: Pretraining and Posttraining Performance Scores 


47 





























































































































"D 

C 

o 

o 

CL 

X 

LD 


CO 

CL 

z 

II 

CNI 

CO 

o 

<: 

CO 

II 


b 

co 


o 

co 

h- 

II 

CO 

O 

CD 

h“ 

II 

CN 


C 

o 

O 

it 


c 

o 

c 

o 

O 


Figure 10: Performance Gain Scores by Experimental Conditions 
and Sites 
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Scatter Plot of Post vs Pretraining Performance Scores 



site: i=swos2=nps Preperf 



Condition: 1=Control 2=Tact 3=Tact+ 


Scatter Plot of Gain vs Pretraining Performance Scores 




2.0 2.5 3.0 

Site: t=SWOS 2=NPS 


3.5 4.0 4.5 5.0 5.5 

Preperf 


Condition: 1=Control 2=Tact 3=Tact+ 


Figure 11: Scatter Plots of Performance Scores by Sites 
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Pretraining Teamwork by Condition and Site 
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Figure 12: Pretraining and Posttraining Teamwork Behavior 
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Figure 13: Teamwork Behavior Gain Scores by Experimental 

Conditions and Sites 
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Scatter Plot of Post vs Pretraining Teamwork Scores 
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V. CONCLUSIONS 


The purpose of this chapter is to discuss the experimental 
results with regard to the two hypotheses presented in Chapter 
II and to revisit the initial research questions. 


A. HYPOTHESES 

Based upon the detailed analysis presented in Chapter IV, 
the two null hypotheses are rejected in favor of the 
alternate hypotheses. In both cases, data gathered during the 
course of the experiment supports the premise that adaptation 

•* i 

and coordination training strategies, when implemented, have 
a measurable effect on a team's ability to perform under the 
test conditions presented during the course of this study. A 
summary table (i.e., Table VII) is presented to show the 
effectiveness of the training strategies across the 
experimental conditions for both of the hypotheses. 


B. RESEARCH QUESTIONS 

The primary research questions from Chapter II were: 

1. If team members are taught to monitor other team members 
and offer assistance during periods of high stress 
(presence of heavy workload on a team member) will the 
teams overall level of performance improve? 

2. If teams are given a technique to achieve a shared 
mental model of the current tactical picture, such as 
structured TAO situation assessments, would their 
performance level be improved? 
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Table VII: Pre-Posttraining Scoring Summary 


MEASURE 

Score: Mean 
(Standard Deviation) 


Control 

Tact 

Tact+ 

Performance 

Pre 

3.5573 

(1.1232) 

3.5615 

(1.1853) 

4.6906 

(0.7109) 


Post 

3.5812 

(1.1139) 

4.2792 

(1.0333) 

5.5240 

(0.3032) 

Performance Gain 

0.0240 

(0.6037) 

0.7177 

(1.1274) 

0.8333 

(0.6632) 

Teamwork 

Pre 

3.4008 

(1.0535) 

3.5383 

(0.9178) 

4.5608 

(0.8671) 


Post 

3.5317 

(0.9236) 

4.2642 

(0.8819) 

5.4225 

(0.5573) 

Teamwork Gain 

0.1308 

(0.5922) 

0.7258 

(1.3967) 

0.8617 

(0.9237) 


Question one was answered in the affirmative within the data 
analysis section of the previous chapter. The experimental 
evidence strongly suggests that once teams are taught 
adaptation and coordination strategies, their overall 
performance level and exhibition of teamwork behavior skills 
are improved upon. 

The second question was not fully explored during the data 
analysis. The focus of the thesis became determining whether 
or not training had an effect on the teams versus no training. 
Therefore, the statistical tests were used to measure 
comparisons to a baseline provided by the control groups vice 
a comparison at each incremental level of training. However, 
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from the scoring summary table, it appears that the additional 
structured information (TAO's sitrep) provided by the tact 
plus training was beneficial to those teams. On the surface, 
they consistently appear to rate higher than the tact groups 
during the experiment. 19 

A third research question was: could the experiment 
provide insight into the decision-making process during 
stressful periods which could be used to further improve team 
performance under the test conditions? During the course of 
the analysis, there was no evidence that the level of stress 
in the scenarios significantly effected the outcomes. 20 Noting 
that stress does not appear to be a significant factor in the 
experimental model, we are left with no clear answers 
regarding performance in low versus high stress scenarios and 
whether the training significantly impacts one training 
condition more than the other. We are left to conclude that 
no additional insight into the decision-making process under 


19 Statistical tests of significance were not conducted 
between the tact and tact plus groups to confirm that the 
differences in their apparent scores were not simply a random 
occurrence. 

20 It is the author's opinion that no experimental factors can 
truly model the stress felt on the high seas. Recognizing this, 
there is probably no need to try to distinguish between the high 
and low stress scenarios presented in this study. A ship's company 
can be just as stressed by a single unknown uncooperative aircraft 
that may be flying a potentially threatening profile as by five 
aircraft and a half dozen surface craft operating in the near 
vicinity which is causing a much greater temporal workload. 
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stressful conditions "per se" was gained as a result of the 


experiment. 
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VI. RECOMMENDATIONS 


The purpose of this chapter is to provide recommendations 
regarding the TACT experiment based upon the finding of the 
data analysis and personal observations of the author. 

A. EXPLORATION OF SCORING DIFFERENCES 

During the course of the experiment, different observers 
were used to rate different teams from the two test sites. As 
a result of looking at the differences in scores from the two 
sites; one is left to question if differences in team scores 
were truly attributable to the test factors or were they 
attributable to the observers. It is recommended that the 
contractor review the criteria for assignment of observers and 
their training to ensure consistency in the subjective 
evaluation of the teams. It is recognized that any form of 
subjective evaluation is just that i.e., "subjective” but it 
is worth reviewing the procedures to ensure that results are 
not tainted due to biased or partially trained observers. 

B. GUARDING AGAINST BIASING THE DECISION-MAKING PROCESS 

Teams that the received the tact plus training generally 

rated higher than the other teams. If the difference in their 
rating is attributable to the TAO's sitreps, then it is also 
imperative that an appropriate counter training be provided to 
ensure that team members are warned that while the sitrep is 
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a tool to share the TAO's view of the tactical picture, it is 
not meant to suppress input. Team members must be alert to 
guard against the confirmatory biasing discussed in Gough's 
Thesis (Gough, 1992). Operators subordinate to the TAO must 
be trained that they are to speak out when they disagree with 
the TAO's analysis for they may be holding a critical piece of 
information that the TAO may not be aware of in formulating 
his situational assessment. 
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APPENDIX A. EXPERIMENTAL MEASURING INSTRUMENTS 

TACT EXPERIMENT (TADMUS) 

CIC TEAM PRE-MISSION QUESTIONNAIRE 
TEAM#_ SIT E DAT E, TEAM POSITIO N SCEN#_ 

1. How much confidence do you place in the ability of the other members of your team to 
accomplish this mission? 

Very Liule 1234567A Great Deal 

2. How much confidence do you think the other team members place in your ability to accomplish 
this mission? 

Very Little 1 2 3 4 5 6 7 A Great Deal 

3. To what extent should team members be aware of other team members workload? 

Very Little 1234567A Great Deal 

4. To what extent do highly competent team members experience stress? 

Very Little 1 2 3 4 5 6 7 A Great Deal 

5. A team member’s decision making ability is as good in stressful situations as it is in non¬ 
stressful conditions. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

6. Monitoring the TAO's performance for possible mistakes and errors tends to reduce the TAO's 
stature and authority. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

7. A team member should offer task help to another team member only if he/she is sure the team 
member needs it. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

8. To what extent should team members monitor other team members for signs of stress? 

Very Little 1 2 3 4 5 6 7 A Great Deal 

9. To what extent should team members mention/share their own feelings of stress/workload with 
other team members during a mission? 

Very Little 1234567A Great Deal 
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10. Even when stressed, I perform effectively during critical aspects of the mission. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

11. To what extent should the team members change their work strategy in response to high 
stress/workload? 

Very Little 1 2 3 4 5 6 7 A Great Deal 

12. Communications among team members are rarely affected by high stress/high workload. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

13. To what extent should team members take account of other team members personalities for 
effective crew coordination? 

Very Little 1 2 3 4 5 6 7 AGreatDeal 

14. To what extent can the effectiveness of crew coordination be lowered by stress/workload? 

Very Little 1 2 3 4 5 6 7 AGreatDeal 

15. It is not a good idea to point out an error committed by a team member during a mission. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

16. To what extent are reprimands more effective than discussions in eliminating some elements of 
a team member’s poor task performance? 

Very Little 1 2 3 4 5 6 7 AGreatDeal 

17. To what extent is understanding the CO’s/TAO's concepts/beliefs of the situation/mission 
important to a team member’s execution of the mission? 

Very Little 1 2 3 4 5 6 7 AGreatDeal 

18. Task overload usually occurs because a team member is not very competent. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

19. Each team member should watch for situations in which external events hinder the performance 
of other team members. 

Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 

20. Team members should be able to anticipate each other’s information needs during the mission. 

Strongly Disagree 1 2 3 4 5 6 7 
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TACT EXPERIMENT (TADMUS) 

CIC TEAM POST-MISSION QUESTIONNAIRE 
TEAM#_ SITE DA TE TEAM POSITIO N SCEN# 


1. How much confidence did you have during the mission that the TAO would successfully 
complete the mission ? 

1 i 2 i 3 i 4 i 3 i * i 7 

Very Little Moderate A Great Deal 

2. How much confidence did you have during the mission that the other team members would 
successfully complete the mission? 

1 i 2 i 3 i 4 i 5 i « i 7 

Very Little Moderate A Great Deal 

3. How much assistance did you provide to other team members as the mission unfolded? 

I 1 i * i 3 i 4 i 5 i 6 i 7 I 

Very Little Moderate A Great Deal 

4. To what extent did you cross-monitor the actions of other team member as the mission 
unfolded? 

I ■ i 2 i 3 i 4 i 5 i « i 7 I 

Very Little Moderate A Great Deal 

5. To what extent were you able to anticipate (i.e., predict) the actions and decisions of the TAO? 



Rarely Half The Time All The Time 

6. To what extent were you able to anticipate (i.e., predict) the actions and decisions of the other 
team members? 

I 1 i 2 i 3 i 4 » 5 i 6 i 7 I 

Rarely Half The Time All The Time 
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7a. What was the most critical episode of this mission? 


b. During this episode to what extent were you thinking and acting “in sync” with the TAO? 



Very Little Moderate A Great Deal 

c. How do you know that? _ 


d. During this episode to what extent were you thinking and acting “in sync” with other team 
members? 


± 


.1 ■ *, I 


1 


Very Little 

e. How do you know that? 


Moderate 


A Great Deal 


Put an "X" on each of the six scales below, at the point that matches best your 
workload experience for the mission you have just accomplished. 


Mental Demand 

L 

Ll 

1 i 

1 , 

1 i 

| | 

| i 

| , 

| , 

1 1 1 

Very Low 








Very High 

Physical Demand 

Li 

Ll 

1 i 

1 i 

1 i 

| 

1 i 

| i 

| i 

1 i 1 

Very Low 








Very High 

Temporal Demand 
(Time Pressure) 

Ll 

Ll 

1 i 

1 i 

1 i 

| | 

1 i 

1 i 

1 i 

1 i 1 

Very Low 








Very High 

Performance 

1 i 1 

1 I 

, I 

| | 

| 

1 I 

1 I 

l 1 

1 1 1 

| | 

Perfect 









Failure 

Effort 

1 . 1 

i J 

l | 

l | 

| | 

l | 

, | 

i | 

1 i 1 

I ( 

Very Low 








Very High 

Frustration 

i. i 

lI 

l | 

, | 

| | 

l I 

, | 

i 1 

l 1 

i 1 

Very Low 








Very High 
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TACT EXPERIMENT (TADMUS) 

TEAMWORK AND PERFORMANCE: OBSERVER’S RATING FORM 

TEAM#_ SITE DA TE OBSERVER_ SCEN#_ 

Instructions for Teamwork Ratings 

Circle a number on the scale accompanying the questions on the following pages so that it best 
describes the behavior of the team you just observed. Consider each team separately. Try not to 
compare one team to another. Instead strive to rate the behavior of a team on an absolute scale. To 
help you perform this absolute rating a brief description of the behavior you should observe for the 
highest rating on the scale and a brief description of the behavior you should observe for the lowest 
rating on the scale are provided for each question. Read these guides or anchors carefully and refer 
to them as you rate the team on each item. Feel free to write comments or explanations for any 
question. 

The rating scales or questions for teamwork are organized into six areas. To further help you 
in your ratings each area is defined below. Please read these definitions carefully. 


Team Orientation 

Team orientation refers to the commitment team members have and exhibit to working together. 
It implies that they place the goals and interest of the team ahead of their personal goals. It also 
refers to the trust each team member has in the other team members, team pride, and esprit de corps. 

Communication Behavior 

Communication involves the exchange of information between two or more team members in 
the prescribed manner and by using proper terminology. Often the purpose of communication is to 
clanfy or acknowledge the receipt of information. 

Monitoring Behavior 

Monitoring refers to observing the activities and performance of other team members. It 
implies that team members are individually competent and that they may subsequently provide 
feedback and backup behavior. 

Feedback Behavior 

Feedback involves the giving, seeking, and receiving of information among members. Giving 
feedback refers to providing information regarding other member’s performance. Seeking 
feedback refers to requesting input or guidance regarding performance. Receiving feedback refers 
to accepting positive and negative information regarding performance. 

Back-up Behavior 

Backup behavior involves assisting the performance of other team members. This implies that 
team members have an understanding of other member’s tasks. It also implies that members are 
willing to give and seek assistance. 

Coordination Behavior 

Coordination refers to team members’ executing their activities in a timely and integrated 
manner. It implies that the performance of some team members influence the performance of other 
team members. This may involve an exchange of information that subsequently influences another 
member's performance. 


scenarios 1-/1 + 
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Team Orientation 

1. To what extent was this team oriented toward teamwork? 



2 


3 


4 


5 


6 



7 Good team orientation could be inferred in a situation where a team member places the goals and interests of 


the team ahead of personal goals. Also may be evident through the display of trust, team pride, and esprit de 
corps, and an awareness that teamwork is important. 

1 Poor team orientation manifests itself when members place their personal concerns above the team's success 
(e.g., disregarding or refusing to follow procedures; arguments, quarrels, and open resentment; and becoming 
upset with a member's performance and either ignoring or harassing that member are evidences of poor team 
orientation). 

2. To what extent were errors caused by inadequate team communication? 


7 Communication within the team was always effective and never responsible for errors or degraded 
performance. 

1 Communication was wholly inadequate and resulted in most of the errors made by the team. 

3. To what extent were errors caused by improper individual actions or decisions? 



3 


2 


5 


l 


4 


6 


7 No actions or decisions of a single team member resulted in errors or poor team performance. 

1 The actions and/or decisions by a single team member very frequendy resulted in errors or poor team 
performance. 

Comments: 

Communication Behavior 

4. How well did team members communicate? 


7 Good communication occurs when team members pass on all important information and clarify intentions 
and planned procedures; members obtain necessary information and acknowledge and repeat messages to ensure 
correctness; members ensure that their messages are received as intended. 

1 Poor communication occurs when team members fail to pass on information or intentions, or pass on 
incomplete communications; members fail to clarify information; members fail to acknowledge other member's 
requests or reports; members disregard proper security procedures for communication; members use improper 
terminology; members tie up the net with irrelevant communications. 


scenarios 1-/1+ 
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5. To what extent did the TAO provide tactical direction or relevant information to other team 
members, without the other team members having to ask for it? 



7 TAO always provided important direction or information to other team members without being asked. 

1 TAO never provided direction or information to other team members unless specifically asked. 

6. To what extent did other team members provide relevant tactical information to the TAO, 
without the TAO having to ask for it? 



7 Other team members always provided important information to the TAO without being asked. 
1 Other team members never provided information to the TAO unless specifically asked. 
Comments:_ 


Monitoring Behavior 


7. To what extent did team members monitor each other’s behavior? 



7 Good monitoring occurs when team members consistently observe the performance of the others to ensure 
the efficiency of the team; members notice and are concerned with the performance of the entire team; one 
member recognizes when other team members perform correctly; members consistently keep track of other 
team members' performance. 

1 Poor monitoring occurs when team members fail to notice other team members' performance on almost all 
occasions; members rarely notice when other team members perform correctly or make a mistake. 

8. To what extent did team members alert each other to impending decisions and actions? 



7 Team members always alerted each other to impending decisions and actions; supporting information was 
actively solicited from other team members. 

1 Team members did not keep each other informed of impending decisions and actions; compromises to 
mission safety or mission effectiveness arose when a team member waited for the other to volunteer significant 
information. 


Comments;. 


scenarios 1-/1+ 
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Feedback Behavior 

9. To what extent did team members provide feedback to one another? 



3 


4 


5 


6 


7 


7 Good feedback behavior occurs when team members go over procedures with one another by identifying 
mistakes and how to correct them; members ask for input regarding mistakes and what needs to be worked on; 
members are corrected for mistakes and incorporate the suggestions in their procedures. 

1 Poor feedback behavior occurs when one or more team members makes sarcastic comments to one or more 
members when the scenario doesn't go as planned; members resist asking for advice and make guesses on 
proper procedures; members reject time-saving suggestions offered by other team members. 

Comments:_ 

Backup Behavior 

10. To what extent did team members provided backup to one another? 


7 Good backup behavior occurs when one team member is having difficulty, makes a mistake, or is unable to 
perform duties, and one or more members steps in to help, ensuring that the activity is completed properly; one 
or more members provide critical assistance without neglecting their own assigned duties; the member having 
difficulty or overburdened displays a willingness to seek assistance rather than struggle and make a mistake. 

1 Poor backup behavior occurs when one or more members fail to provide assistance to another member who 
is having difficulty, makes a mistake, or is unable to perform his duties; while providing assistance, the 
members tends to neglect their own duties; members are unwilling to ask for help even when it is available; 
one member provides needed assistance, but does not inform others that he is occupied assisting another or 
what he has done; one member displays an unwillingness to help others even when asked. 

11. To what extent did the TAO anticipate the need to provide (some) assistance to one or more 
team members? 


7 TAO consistently anticipated the need to provide assistance to other team members during critical phases of 
the mission. 

1 TAO never anticipated the need to provide assistance to other team members during critical phases of the 
mission; the other team members always had to ask. 


To what extend did the other team members anticipate the need to provide assistance to the 


12. T 
TAO? 




5 


2 


3 


4 


6 


7 Other team members consistently anticipated the need to provide assistance to the TAO during critical 
phases of the mission. 

1 Other team members never anticipated the need to provide assistance to the TAO during critical phases of 
the mission; the TAO always had to ask. 


scenarios 1-/1+ 
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13. Did the team members adjust individual task responsibilities to prevent overload? 



7 Team members were consistently aware of each other's workload buildup and reacted quickly to adjust 
division of task responsibilities to redistribute workload . 

1 Team members were generally unaware of each other's workload buildup; little or no auempt was made to 
adjust the distribution of task responsibilities before significant compromises to mission safety or mission 
effectiveness occurred. 

Comments:. . . 

Coordination Behavior 

14. To what extent was the team’s behavior coordinated? 

I ' I 3 I 3 I * I 3 I ‘ I 7 I 

7 Good coordination behavior occurs when team members consistently pass critical information to the other 
members, thereby enabling them to accomplish tasks; members consistently cany out tasks quickly or in a 
timely manner enabling others to carry out their tasks effectively. Team members appear very familiar with 
the relevant parts of one another’s jobs and carry out individual tasks in a synchronized manner. 

1 Poor coordination behavior occurs when team members consistently cany out their tasks ineffectively, 
leading to other team members' failing at their tasks; members carry out their tasks unpredictably, leading to 
delays in execution of critical tasks; members neglect to pass on critical pieces of information to one another, 
leading to breakdowns in team performance; team members carry out their tasks with significant delays leading 
to team errors. 

15. How congruent/similar were the TAO’s and the other team members’ understanding of the 
mission? 


I 1 2 1 3 I 4 I 3 I 6 I 7 

7 TAO and other team members were completely in agreement (i.e., congruent) on goals, tasks, and concepts 
involving the mission. 

1 TAO and other team members were rarely in agreement (i.e., congruent) on goals, tasks, and concepts 
involving the mission. 

Comments;_ 
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OVERALL AAW TEAM PERFORMANCE ASSESSMENT 
Instructions for Performance Ratings 

Please assess the performance of the team for the following tasks and/or activities using the 
scales provided. Note that a score of 7 always denotes effective or superior performance, while a 
score of 1 always denotes ineffective or very poor performance. The anchors or descriptors 
associated with the high and low scores are what you should expect to see for very effective and 
very ineffective team performances, respectively. They are provided as guidance for your ratings. 

1. Making radar detection repons . This refers to the report made by any team member who 
verbally describes the radar contact. 

Very Poor 1 2 3 4 5 6 7 Superior 

7 The radar detection reports are always accurate, concise, and timely. Proper format (e.g., APP-1, NWP-32) and 
terminology are always used. 

1 Some radar detections are never reported. Many reports are inaccurate and late. Often proper format and 
terminology are qqi used. 

2. Making ESM detection repons . This refers to verbal repons of ESM detections. 

\ 

Very Poor 1 2 3 4 5 6 7 Superior 

7 The ESM detection reports are always accurate, concise, and timely. Proper format and terminology are always 
used. 

1 Some ESM detections are never reported. Many reports are inaccurate and late. Often proper format and 
terminology are mu used. 

3. Identification/Correlation repons . This refers to verbal reports of the correlation and 
identification of contacts. 

Very Poor 1 2 3 4 5 6 7 Superior 

7 The ID/Corretaiion reports are always accurate, concise, and timely. Proper format and terminology are always 
used. 

1 Some ID/Correlations are never made and/or reported. Many reports contain errors and/or are late. Frequently 
improper format and incorrect terminology are used 

4. Assessment of contacts’ hostile intent . This is typically based on input from lower levels 
within the team and made by the TAO or CO. 

Very Poor 1 2 3 4 5 6 7 Superior 

7 TAO/CO routinely assess the threat of each new contact and advise the rest of the team accordingly. Assessment 
is firmly based on information the team has collected (e.g., ESM, ID/Corr, Intel) and on verbal discussions 
concerned with weapons loads, flight profiles, and attempts at communication with the contact. 

1 TAO/CO infrequently assess the threat of new contacts and/or rarely advise the rest of the team as to the 
contact's threat. Assessment is often not based on available information and verbal discussion about such aspects as 
weapons load and flight profile have not occurred. 
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5. Monitoring the threat . This pertains primarily to critical contacts of interest (CCOI). 

Very Poor 1 2 3 4 5 6 7 Superior 

7 CCOIs arc frequently hooked and observation of them are more or less constant. The status of the CCOIs are 
frequently discussed and appraised - in short the intensity of involvement with these threats is high. 

1 CCOIs are frequently neglected or overlooked. The status of CCPIs are not reviewed, discussed, or appraised 
frequently enough - in short, the intensity of involvement with these threats is low. 

6. Taking appropriate acti on in accordance with ROE. This refers to whether the team decides to 
take some action against a given CCOI vs. failure to do anything about it 

Very Poor 1 2 3 4 5 6 7 Superior 

7 TAO (or CO) and team consistently take effective and appropriate actions to deal with threats. This includes 
assigning CAP, covering, issuing verbal warnings, increasing readiness/going to GQ, activating doctrine, and 
determining chaff solutions. 

1 TAO (or CO) and team are lax and often fail to take effective or appropriate actions to deal with threats. They 
tend to over react or fails to react 

7. Planning for the upcoming mission. This refers to all planning activities performed by the TAO 
or other team members for the upcoming mission. 

Very Poor 1 2 3 4 5 6 7 Superior 

7 The TAO and/or other team members spend a reasonable amount of time planning for the upcoming mission. 
Roles are further defined and tasks that are outside normal responsibility assigned. Critical events that might occur 
are clearly defined and specific responses agreed upon. 

1 The TAO and/or other team members spend little or no time planning for the upcoming mission. Roles are not 
further defined and tasks that are outside normal responsibility are not assigned. Little or no discussion occurs about 
critical events that might occur. Those events that are mentioned are not defined well nor are responses to the events 
delineated. 


8. Overall performance rating of this team for this scenario . 

Very Poor 1 2 3 4 5 6 7 Superior 

7 Superior teams have consistently scored well on the above six areas, as well as on other unassessed areas. 
1 Poor teams have consistently scored poorly on the above six areas, as well as on other unassessed areas. 
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9. Performance of critical events. Below are four critical events that occurred in this scenario . 
Rate how the team performed each on the seven point scales provided. 

a. Four Iranian F4s detected. 



Very Poor 1 2 3 

4 

5 

6 

7 

Superior 

b. 

Iranian bogies split into two sections. 







Very Poor 1 2 3 

4 

5 

6 

7 

Superior 

c. 

APQ120 detected (Iranian F4). 







Very Poor 12 3 

4 

5 

6 

7 

Superior 

d. 

Low F4s pop-up at 46nm. 







Very Poor 12 3 

4 

5 

6 

7 

Superior 
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TACT EXPERIMENT (TADMUS) 

CIC TEAM COMMUNICATION RECORDING FORM: TAO 
TEAM#_ SUE_ DAT E OBSERVE R SCEN#_ PERIOD 1 2 3 
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TACT EXPERIMENT (TADMUS) 

CIC TEAM COMMUNICATION RECORDING FORM: Team 
TEAM#_ SITE_ DATE OBSERVER SCEN#_ PERIOD 1 2 3 



Additional notes / Other categories: 


72 















































TACT DATA SPREADSHEET 


APPENDIX B 


DATA SPREADSHEETS 


o ; 

U> 

r> 

n 

in 

ri 

n 

CM 

co 

ri 

in 

ri 

ri 

in 

ri 

TT 

m 

ri 

® 

in 

GO 

in 

CM 

in 

in 

ri 

CM 

V3 

in 

ui 

r» 

<d 

o 

in 

Cl 

CM 

CM 

in 

XT 

in 

CM 

<n 

U) 

in 

XT 

in 

XT 

in 

XT 

XT 

x» 

in 

ri 

XT 

in 

XT 

in 

W 

ui 

ri 

m 

ri 

Ui 

xf 

XT 

Ui 

Ui 

ri 

in 

xf 

in 

ri 

Ui 

xf 

ID 

ui 

xf 

Xf 

c 

o. 

O 

V 

n 

X 

i 

E 

r> 

xf 

GO 

CM 

00 

CM 

r> 

in 

CM 

o 

in 

ri 

in 

xf 

xt 

n 

in 

® 

in 

00 

CM 

r> 

ri 

in 

to 

in 

in 

r> 

CD 

r> 

<o 

in 

CM 

CM 

in 

XT 

Ui 

ui 

in 

ri 

in 

ri 

cn 

r> 

cn 

cn 

in 

ri 

XT 

cn 

in 

CM 

XT 

'T 

cn 

lO 

CM 

in 

ri 

Ui 

ri 

u> 

ri 

Ui 

ri 

ip 

CM 

u> 

ri 

in 

xf 

m 

XT 

U) 

ri 

o ! 

x» 

D 

* 

i 

E 

in 

ri 

ri 

ri 

m 

ri 

n 

ri 

00 

CM 

in 

CM 

CD 

CM 

CO 

ri 

xt 

r> 

id 

m 

cb 

in 

cm’ 

CM 

in 

ri 

CM 

to 

to 

Cl 

to 

co 

ui 

GO 

CM 

CM 

n 

XT 

cn 

ui 

in 

CM 

Ui 

ri 

xT 

Ui 

in 

XT 

n 

in 

xT 

Ui 

XT 

XT 

Ui 

XT 

u> 

nr 


to 

ui 

op 

xf 

in 

V 

XT 

u> 

ri 

ip 

Ip 

ui 

in 

u> 

in 

m 

xf 

Z 

in 

xf 

»» 

00 

ri 

m 

ri 

in 

CM 

in 

'J 

in 

ri 

in 

CM 

Cl 

xr 

® 

ri 

® 

in 

00 

in 

cm 

ui 

n 

ri 

CM 

in 

in 

to 

r> 

CO 

in 

CM 

r> 

CM 

® 

xf 

in 

CM 

in 

cm’ 

m 

in 

XT 

xf 

m 

x» 

in 

XT 

T 

* 

V 

Ui 

xT 


cn 

in 

ri 

in 

m 

XT 

ID 

ri 

in 

in 

ri 

in 

u> 

u> 

xf 


xf 

D 

xt 

xf 

m 

ri 

m 

ri 

n 

ri 

in 

ri 

n 

CM 


n 

ri 

in 

in 

® 

in 

CM 

n 

CM 

n 

ri 

Cl 

in 

v> 

Cl 

to 

n 

ui 

CM 

cn 

CM 

u> 

XT 

m 

cn 

u> 

CM 

in 

ui 

xT 

Ui 

XT 

XT 

in 

Ui 

XT 

Ui 

ri 

ID 

XT 

in 

ui 

XT 

cn 

cn 

in 

in 

ID 

ID 

ui 

m 

xf 

up 

xf 

m 

ip 

xf 

in 

ui 

® 

in 

in 

ui 

ti 

-j 

u> 

ci 

a 

tt 

i 

E 

r> 

r> 

rj 

CM 

r> 

'T 

® 

CO 

CM 

r> 

CM 

00 

xf 

in 

ri 

ri 

<b 

CO 

CM 

® 

CM 

ri 

ri 

n 

CM 

® 

ri 

xt 

in 

ui 

in 

XT 

cn 

ri 

cn 

CM 

cn 

ui 

<D 

to 

to 

CO 

in 

to 

in 

ui 

m 

in 

ui 

in 

to 

Pv 

Ui 

to 

in 

to 

xr 

Ui 

ri 

to 

CO 

Ui 

xf 

x» 

Ip 

xf 

Ui 

xf 

U) 

f 

m 

00 

ib 

ID 

* 

00 

ri 

T 

tf 

3 

6 

in 

ri 

n 

ri 

n 

n 

in 

CM 

in 

CM 

CM 

xf 

ri 

r> 

(O 

<o 

® 

in 

® 

CM 

00 

in 

in 

in 

in 

r> 

CO 

in 

ui 

00 

CM 

XT 

m 

Ui 

ri 

Ui 

ri 

in 

xT 

Ui 

V 

in 

xf 

U) 

XT 

in 

Ui 

in 

Ui 

ui 

ui 

in 

n 

ID 

W 

xf 

<D 

in 

ui 

Ui 

ui 

XT 

in 

u> 

W 

ip 

ID 

in 

ui 

in 

u> 

to 

-> 

00 

ri 

n 

E 

® 

ri 

® 

r>i 

in 

CM 

n 

CM 

r> 

CM 

CM 

to 

m 

c- 

00 

ri 

CO 

in 

Cl 

in 

n 

CM 

GO 

Cl 

ri 

Cl 

CO 

in 

in 

Cl 

to 

CO 

XT 

n 

CM 

cn 

CM 

in 

XT 

in 

m 

XT 

in 

XT 

xT 

in 

ri 

m 

ri 

XT 

in 

ri 

Ui 

ri 

m 

® 

ri 

in 

XT 

Ui 

ri 

Ui 

ri 

cn 

in 

ui 

in 

f 

in 

xf 

XT 

Ui 

ri 

u> 

ui 

in 

ui 

ID 

id 

xf 

C 

00 

IM 

N 

-C 

| 

r> 

ri 

00 

CM 

ui 

CM 

in 

CM 

r> 

CM 

in 

CM 


| 

r> 

® 

in 

m 

00 

CM 

® 

! CM 

CO 

«o 

in 

in 

to 

cn 

ui 

® 

CM 

cn 

xT 

m 

XT 

in 

XT 

xT 

cn 

u> 

ri 

xT 

XT 

in 

XT 

Ui 

U> 

W 

XT 

XT 

m 

XT 

XT 

in 

ui 

ID 

xf 

ID 

nf 

Ui 

xf 

ID 

u> 

xf 

in 

id 

ip 

u> 

ID 

in 

m 

Z ’ 

i 

xt 

E 

x» 

® 

ri 

00 

CM 

m 

ri 

® 

CM 

r> 


Cl 

xt 

in 

XT 

Cl 

to 

r> 

<c> 

00 

CM 

CM 

Cl 

ri 

in 

CM 

in 

in 

® 

ui 

r> 

co 

00 

ui 

cn 

CM 

in 

CM 

CD 

XT 

cn 

ui 

cO 

to 

in 

Ui 

in 

XT 

Ui 

XT 

in 

lO 

ui 

u> 

cn 

ui 

ID 

in 

Ui 

ui 

Ui 

ID 

xf 

cn 

in 

lO 

Ui 

ui 

u> 

in 

in 

xf 

in 

to 

ui 

in 

in 

m 

ib 

ip 

ib 

4 

* 

• 

5 

CM 

xf 

a 

CM 


r» 

XT 

CM 


rj 



CM 

Xf 

Cl 


CM 

XT 

cn 

CM 


rn 

xf 


CM 

XT 

cn 

CM 


cn 

XT 

CM 


n 

XT 

CM 


cn 

* 


CM 

Xf 

cn 


CM 

Xf 

n 

-J 

| 

I 

> 

A 

5 

A 

CM 


CM 


' CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


CM 


<M 


CM 

j 

UJ ; 

( 

< 

t 

CM 

? 

i 

> 

l 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 


CM 

CM 


CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

Q i 
* 

C 

> 

X 

j 

X 


CM 

CM 



CM 

1 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 



CM 

CM 

£ 

* 

» 

i 

| 




CM 

CM 

CM 

CM 

r> 

Cl 

Cl 

n 


Xf 

XT 

XT 

m 

in 

in 

m 

CO 

to 

to 

<0 





CM 

CM 

CM 

CM 

r> 

cn 

cn 

<n 


XT 

XT 

* 

in 

® 

in 

in 

« 

e 

« 

to 

X 

c 

* £ 

c 

1 

* 

CN 

1 

i 

i 

i 

CM 

CM 

CM 





rj 

ri 

n 

Cl 





r> 

ri 

o 

m 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 





CM 

CM 

CM 

CM 

cn 

Cl 

cn 

cn 





cn 

ri 

n 

r> 

< 

« 

i 

{ j 

i | 
























CM 

CM 

CM 

CM 

CM 

CM 

CM 

«M 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 


■ CM 

PI 

XT 



r*> 

«0 


o 


CM 

n 

* 

in 


r** 

* 


s 

r* 

CM 

CM 

" 

a 

a 

a 

r» 

CM 

8 

s 

3 

cn 

s? 

8 

a 

a 

a 

P* 

cn 

a 

a 

3 

5 

Sf 

3 

3 

3 

9 

3 

9 

3 


73 




































































































TACT DATA SPREADSHEET 


I 

< 

s 

perf 9d I 

CN 


CN 

if) 

If) 

cn 

■* 

in 

CN 

CN 

in 

in 

if) 

ri 

if) 

XT 

CD 


If) 

CN 

cn 

ifi 

cn 

cn 

CD 

CD 

r) 

ib 

CD 

If) 

ri 

cn 

ri 

cn 

ui 

if) 

if) 

m 

m 

ui 

in 

ui 

op 

U) 

ib 

ID 

in 

ri 

m 

in 

CN 

00 

ui 

nr 

in 

9 

cn 

in 

ri 

if) 

ib 

in 

9 

9 

CD 

to 

ri 

ID 

m 

m 

cb 

to 

o 

05 

5 

a 

cn 

ID 

CO 

If) 

r> 

if) 

cn 

in 

in 

CN 

in 

cn 

ri 

in 

n 

to 

9 

if) 

CN 

*r 

*r 

a> 

rsi 

in 

ID 

cn 

CO 

in 

ri 

in 

CN 

cn 

in 

CD 

9 

m 

nr 

u> 

CD 

nr 

If) 

nr 

cn 

in 

ID 

n 

U) 

in 

in 

IT 

CD 

ui 

if) 

ib 

in 

ui 

cn 

in 

IO 

in 

9 

to 

ri 

m 

in 

to 

ib 

in 

ri 

m 

ui 

ID 

CD 

u. 

« 

J3 

CD 

T 

£ 

CN 

if) 

CN 

if) 

CN 

CN 

in 

in 

in 

in 

ri 

in 

CN 

9 

in 

in 

CO 

CN 

in 

cn 

cn 

CN 

in 

nr 

n 

nr 

cn 

rr 

cn 

CN 

op 

00 

ri 

CD 

ri 

m 

ri 

in 

CN 

CP 

9 

cn 

cn 

ui 

cn 

cn 

in 

U) 

9 

in 

nr 

cn 

in 

in 

to 

ri 

cn 

in 

ui 

to 

m 

ri 

in 

CN 

in 

in 

9 

nr 

cn 

in 

w 

id 

< 

O) 

& 

cn 

cn 

If) 

O 

■tf 

If) 

CN 

cn 

in 

cn 

* 

in 

CN 

*9 

cn 

ifi 

n 

CD 

CO 

CN 

in 

CD 

nT 

cn 

<b 

cn 

to 

CN 

op 

u> 

ri 

in 

ui 

in 

in 

in 

ri 

in 

ui 

in 

9 

in 

ui 

in 

9 

in 

9 

NT 

cn 

cn 

ib 

to 

CN 

u> 

to 

9 

in 

nr 

to 

to 

to 

ib 

cn 

n 

nr 

CD 

N 1 

ID 

cb 

m 

§ 

CO 

1 

CO 

CN 

ip 

CN 

CD 

CN 

cn 

CN 

cn 

00 

CN 

cn 

cn 

CN 

cn 

9 

cn 

ri 

<o 

in 

in 

00 

op 

If) 

ri 

CN 

op 

CD 

in 

in 

m 

CD 

to 

If) 

CN 

CN 

in 

nr 

in 

<<7 

nr 

cn 

ui 

to 

9 

cn 

n- 

9 

cn 

nr 

in 

nr 

in 

9 

cn 

ui 

to 

to 

nr 

9 

cn 

cb 

CD 

ib 

in 

m 

9 

CD 

9 

in 

nr 

CO 

CD 

CD 

ID 

ID 

< 

»n 

I 

CN 

CN 

LO 

CN 

00 

CN 

CD 

CN 

in 

cn 

cn 

CN 

m 

m 

ri 

to 

00 

ui 

CN 

00 

CD 

CN 

cn 

CN 

If) 

in 

i/i 

m 

CO 

CD 

If) 

CN 

op 

r> 

ui 

if) 

ui 

in 

ri 

in 

9 

CD 

to 

U) 

If) 

ui 

ao 

9 

to 

9 

to 

in 

io 

in 

ui 

in 

ui 

m 

ri 

to 

CD 

nr 

CD 

nr 

to 

ri 

to 

ri 

cn 

9 

ID 

9 

m 

ID 

V 

to 

9 

CD 

« 

co 

i» 

a, 

If) 

cn 

n 

If) 

ri 

00 

00 

ci 

m 

ri 

9 

m 

in 

00 

CN 

to 

if) 

m 

CN 

If) 

CN 

in 

ri 

If) 

CO 

in 

ui 

m 

cb 

op 

ui 

r> 

If) 

CN 

in 

nr 

to 

ui 

if) 

ib 

If) 

ui 

ID 

lb 

to 

in 

ui 

in 

ui 

to 

in 

9 

CO 

CP 

up 

in 

cn 

9 

up 

ib 

00 

U) 

to 

to 

9 

in 

ui 

to 

ib 

to 

to 

in 

U) 

u> 

ui 

CD 

ui 


in 

I 

CN 

CN 

in 

cn 

cn 

<N 

cn 

cn 

CN 

<n 

CN 

nT 

CD 

CN 

If) 

in 

in 

m 

If) 

CN 

If) 

CN 

cn 

ri 

CD 

in 

ifi 

in 

m 

cn 

cb 

in 

in 

m 

CN 

OO 

m 

9 

CD 

9 

m 

ui 

m 

ui 

in 

m 

nr 

up 

ib 

in 

to 

ri 

cn 

n 

9 

cn 

in 

in 

cb 

9 

in 

ri 

in 

9 

CD 

CO 

in 

9 

to 

ri 

in 

in 

to 

to 

ib 

co 

in 

ib 

ID 


I 


n* 

°o 

CN 

CN 

cn 

00 

CN 

cn 

CO 

in 

n 

ri 

to 

in 

f) 

00 

If) 

op 

CN 

00 

cn 

co 

in 

ui 

in 

cb 

cn 

co 

00 

CN 

CN 

ao 

■9 

cn 

ui 

u> 

if) 

nr 

in 

ui 

if) 

ui 

in 

ib 

9 

m 

9 

in 

r> 

CD 

ib 

co 

in 

m 

ri 

to 

tf> 

ib 

if) 

ui 

to 

9 

in 

ui 

9 

to 

ib 

m 

ID 

ui 

no 

> 

cn 

£ 

a 

cn 

CN 

lO 

rj 

If) 

CN 

if> 

CN 

cn 

CN 

in 

ri 

o 

nr 

*9 

to 

op 

in 

00 

op 

-9 

QD 

CO 

in 

in 

if) 

cb 

cn 

to 

cn 

CN 

CN 

1 

i 

CD 

9 

if) 

9 

in 

ri 

nr 

in 

CN 

m 

ri 

in 

ri 

cn 

n 

9 

to 

cn 

ui 

to 

9 

9 

in 

If) 

ib 

co 

CD 

nr 

m 

ui 

to 

ri 

* 

r) 

ID 

ui 

in 

ib 

« 

ID 

9 

X 

CN 

V 

a 

in 

If) 

CN 

9 

U) 

cn 

in 

CN 

cn 

in 

CN 

cn 

nr 


00 

•9 

to 

00 

ifi 

cn 

CN 


if) 

nr 

<n 

in 

th 

in 

in 

co 

to 

cn 

IU) 

i 

i 

if) 

ri 

00 

ri 

in 

ui 

in 

in 

co 

If) 

•n 

m 

nr 

m 

ui 

in 

ui 

U) 

ib 

to 

ui 

If) 

9 

to 

tO 

9 

in 

ib 

to 

to 

in 

cb 

to 

<0 

in 

ib 

<o 

\ 

1 

to 

9 

in 

■ur 

m 

to 

5 

f 

a. 

CD 

CN 

n 

m 

cn 

CO 

CN 

in 

CN 

in 

CN 

CN 

CO 

ri 

in 

ri 

9 

in 

in 

CN 

CN 

cn 

ri 

CN 

nr 

in 

cn 

cb 

U) 

U) 

m 

CN 

! CN 

! 

| 

if) 

9 

if) 

in 

CN 

cn 

cn 

nr 

nr 

cn 

cn 

U) 

nr 

n 

ID 

nr 

9 

to 

ib 

if) 

9 

m 

9 

U) 

to 

in 

in 

to 

in 

up 

9 

in 

9 

to 

9 

ID 

in 

CD 

m 

CP 

to 

to 

> 

in 

j t 

i 

E 

00 

CN 

CD 

ri 

cn 

CN 

cn 

cn 

N- 

cn 

CN 

cn 

cn 

CN 

in 

m 

ri 

cn 

CD 

00 

ifi 

CN 

00 

cn 

ri 

00 

CD 

if) 

m 

ui 

cn 

cb 

If) 

ui 

cn 

oi 

CN 

in 

ui 

if) 

ib 

if) 

U) 

m 

in 

9 

m 

nr 

m 

n 

in 

nr 

in 

9 

in 

in 

9 

m 

ui 

to 

to 

to 

9 

9 

IO 

to 

in 

to 

U) 

in 

in 

ib 

LO 

ib 

ID 

ui 

to 


-*c 

? 

E 

if) 

ri 

CO 

CN 

00 

ri 

CN 

in 

ri 

00 

CN 

cn 

n 

CN 

cn 

nr 

cn 

cn 

to 

m 

in 

00 

CN 

CD 

ri 

CD 

CD 

in 

in 

ui 

cn 

cb 

CD 

cn 

CN 

If) 

op 

9 

u> 

ui 



m 

9 

n 

9 

in 

9 

in 

m 

to 

ui 

IO 

to 

to 

If) 

9 

in 

9 

9 

CO 

to 

in 

in 

ib 

up 

9 

up 

nr 

ID 

to 

ib 

ID 

« 

co 

CD 

ui 

►» 

cn 

j< 

5 

E 

CO 

CN 

CO 

ri 

cn 

CN 

in 

cn 

<n 

cn 

CN 

m 

*9 

O' 

i 

to 

f) 

ifi 

CD 

if) 

r> 

ri 

in 

CN 

co 

in 

in 

cn 

co 

cn 

cb 

CN 

If) 

CO 

9 

cn 

ui 

in 

IN 

cn 

If) 

9 

m 

n» 

9 

in 

ri 


cn 

m 

nr 

9 

to 

cn 

rn 

nr 

U) 

9 

nr 

9 

to 

9 

nr 

* 

up 

nr 

up 

V 

9 

to 

CN 

? 

E 

if) 

ri 

9 


If) 

CN 

oo 

ri 

in 

ri 

m 

9 

CD 

CN 

ml 

nr 

CD 

ri 

cn 

tfi 

<© 

cn 

CN 

CN 

ao 

ri 

CN 

op 

ifi 

cn 

<D 

cn 

cb 

cn 

cb 

If) 

CN 

CN 

in 

n» 

U) 

in 

ri 

<n 

If) 

V 

nr 

up 

<N 

in 

ri 

to 

ri 

m 

ri 

cn 

* 

9 

9 

nr 

cn 

nf 

to 

9 

If) 

to 

ri 

9 

to 

9 

m 

nf 

u> 

<D 

9 

CC 

-X 

i 

E 

CP 

CN 

r> 

CN 

CN 

CO 

ri 

if) 

CN 

in 

CN 

00 

CN 

<9 

00 

ri 

cn 

CO 

in 

ifi 

If) 

00 

m 

CN 

cn 

9 

m 

ifi 

cn 

cb 

op 

9 

m 

CN 

CN 

m 

9 

u> 

ib 

if) 

9 

m 

If) 

U) 

ui 

U) 

ib 

in 

nr 

9 


in 

ui 

cn 

ui 

to 

ui 

to 

9 

m 

CN 

in 

to 

9 

to 

ri 

to 

cn 

9 

9 

V 

nr 

nr 

9 


- 

CN 

CO 

nT 

If) 

<© 

f- 

00 

0) 

O 

*- 

CN 

m 

nf 

ID 

10 

r* 

5 

0) 

8 

CN 

IN 

CN 

8 

8 

8 

8 

r- 

CN 

8 

8 

8 

cn 

CN 

cn 

8 

8 

8 

8 

r- 

« 

8 

8 

t 

9 

3 

3 

3 

S 

3 

rv 

nr 

32 


74 




















































































MODIFIED DATA SPREADSHEET 


M 

C 

•H 

<TJ 

O' 

* 

I 

o 

vO 

in 

• 

o 

1 

1.4133 

0.1667 

v£> 

ro 

o 

i 

VO 

ro 

o 

i 

*3* 

co 

Ol 

0.7533 

in 

d 

O' 

VO 

*T 

O' 

• 

rH 

1 0.6933 1 

Ol 

CN 

rH 

CO 

C0 

C^ 

CN 

• 

o 

CN 

rH 

o 

i 

CN 

rH 

o 

CO 

ro 

CO 

CN 

• 

o 

[ 0.1667 | 

vO 

as 

• 

o 

1 

ro 

co 

rH 

CN 

co 

| 0.4333 | 

CO 

•H 

rH 

• 

O 

1 

co 

co 

as 

rH 

• 

CN 

<3 

O 

. 

o 

1 

CN 

CO 

• 

rH 

CO 

rH 

in 

• 

o 

1 

83 

1 

4J 

<P 

W 

0 

04 

o 

vO 

00 

00 

CM 

• 

rn 

m 

CO 

ci 

co 

C0 

ro 

ro 

rH 

04 

. 

CO 

CO 

CO 

in 

co 

Kr 

co 

CO 

in 

in 

• 

«r 

O' 

VO 

vO 

00 

cO 

Ol 

o 

# 

vO 

r- 

v0 

04 

CN 

VO 

(N 

rH 

• 

in 

vO 
O 
CN 
• , 

in 

vO 

♦ 

CN 

04 

o 

• 

04 

ro 

co 

co 

ro 

• 

co 

<3* 

ro 

ro 

co 

vo 

04 

VO 

CN 

• 

in 

• 

CO 

CO 

CO 

in 

# 

VO 

O' 

• 

in 

co 

co 

co 

in 

in 

o 

VO 

CO 

O' 

*3 

CN 

O' 

• 

0 

M 

i 

0 

U 

04 

co 

co 

in 

■*3* 

ro 

o* 

vO 

CN 

o 

04 

r- 

10 

lO 

rH 

O' 

cO 

10 

cO 

ro 

co 

o* 

in 

co 

CO 

CO 

rH 

CO 

• 

Ol 

00 

CO 

r- 

vO 

vo 

CO 

• 

<3- 

00 

04 

t 

co 

CO 

co 

in 

in 

a 

co 

CO 

co 

CO 

as 

• 

VO 

O' 

CN 

as 

rH 

rH 

CO 

CO 

co 

rH 

. 

co 

co 

as 

in 

co 

o* 

VO 

o 

CN 

o- 

vo 

VO 

as 

♦ 

m 

o- 

vo 

■*r 

VO 

• 

O' 

vO 

VO 

in 

co 

co 

co 

O' 

in 

• 

m 

O' 

vO 

vO 

• 

co 

co 

co 

CO 

CN 

• 

in 

1*4 

c 

•H 

(0 

ay 

HH 

u 

0) 

04 

ro 

00 

rH 

o 

O' 

rH 

0^ 

CN 

• 

r-i 

in 

• 

o 

i 

o* 

vo 

Ol 

o 

i 

in 

co 

* 

o 

rH 

as 

in 

rH 

co 

CO 

00 

vo 

• 

o 

in 

CN 

rH 

rH 

O' 

vO 

VO 

as 

• 

o 

co 

00 

in 

O' 

t 

o 

in 

<N 

*3* 

rH 

in 

CN 

• 

o 

00 

in 

•VT 

o 

I 

CO 

• 

o 

1 

in 

tH 

• 

o 

ro 

00 

in 

vf 

o 

in 

rH 

. 

o 

1 

CO 

CO 

CO 

CO 

• 

CN 

O' 

VO 

vO 

O 

o 

00 

in 

O' 

• 

o 

i 

co 

00 

in 

o 

• 

CN 

in 

O' 

co 

• 

o 

co 

CO 

o 

00 

o 

in 

CN 

o 

• 

o 

w 

MH 

u 

0) 

04 

(ft 

0 

04 

ro 

00 

o 

o* 

CN 

O' 

vO 

vo 

in 

co 

o* 

CO 

cO 

CO 

<3* 

in 

Ol 

in 

CO 

00 

o 

o 

co 

ro 

00 

o 

04 

CO 

co 

00 

rH 

in 

in 

o 

co 

CO 

co 

ro 

co 

in 

co 

00 

o 

04 

vO 

co 

vO 

in 

CO 

00 

in 

’vr 

in 

rH 

04 

CN 

co 

00 

o 

o 

04 

in 

vo 

rr 

C0 

co 

00 

in 

m 

O' 

•<3 

• 

ro 

00 

in 

as 

o 

*H 

co 

co 

co 

in 

TT 

in 

O' 

in 

m 

CN 

vo 

♦ 

in 

CO 

CO 

CO 

<3 

• 

in 

r* 

rH 

as 

tH 

• 

in 

Q 

*4-1 

& 

Q) 

U 

04 

O' 

rH 

O' 

00 

04 

in 

o* 

Ol 

• 

04 

O' 

cO 

cO 

00 

r- 

rH 

as 

O' 

. 

CO 

00 

in 

vo 

• 

CN 

VO 

rH 

CJO 

• 

04 

in 

in 

04 

O' 

vo 

vO 

co 

in 

in 

O' 

rH 

<3* 

o 

** 

co 

00 

o 

CN 

• 

in 

o 

04 

ro 

00 

O 

co 

CN 

in 

in 

CN 

rH 

• 

in 

Ol 

vO 

CN 

in 

CN 

rH 

04 

in 

O' 

co 

• 

O' 

«H 

*3 

co 

in 

o* 

vO 

rH 

•<* 

• 

CO 

m 

CN 

in 

in 

CN 

vo 

• 

O' 

vO 

vO 

tH 

• 

in 

U 

QJ 

4J 

-H 

W 

rH 

rH 

<N 

Ol 

«H 

rH 

Ol 

Ol 

rH 

rH 

04 

CN 

rH 

rH 

04 

CN 

rH 

rH 

CN 

CN 

tH 

*H 

CN 

CN 

S3 

T3 

C 

0 

0 

Oil 

X 

w 

rH 

rH 

*H 

tH 

Ol 

Ol 

Ol 

Ol 

CO 

co 

CO 

CO 

«H 

rH 

rH 

rH 

CN 

CN 

04 

CN 

CO 

CO 

Cl 

CO 

< 

(ft 

(ft 

Q) 

o 

4J 

(/) 

rH 

rH 

rH 

rH 

rH 

rH 

tH 

rH 

rH 

rH 

rH 

rH 

CN 

CN 

04 

CN 

CN 

04 

CN 

CN 

CN 

CN 

CN 

CN 


rH 

CN 

CO 

H* 

in 

VO 

o 

00 

OV 

o 

rH 

rH 

rH 

04 

rH 

CO 

*H 

rH 

in 

rH 

vo 

•H 

O' 

rH 

CO 

rH 

o\ 

<H 

Q 

CN 

rH 

CN 

CN 

CN 

CO 

04 

*3 

CN 

m 

CN 


75 















































cn 


Normal Probability Plot 

Pretraining Performance Data 



Average: 3.93646 
StdDev: 1.12344 
N of data: 24 


Preperf 


Anderson-Darling Normality Test 
A-Squared: 0.957 
p-value: 0.013 
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Normal Probability Plot 

Posttraining Performance Data 



Average: 4.46146 
Std Dev: 1.18500 
N of data: 24 


Postperf 

Anderson-Darling Normality Test 
A-Squared: 1.069 
p-value: 0.007 

































Normal Probability Plot 

Performance Gain Data 



Average: 0.525 
Std Dev: 0.874594 
N of data: 24 


Perfgain 


Anderson-Darling Normality Test 
A-Squared: 0.368 
p-value: 0.401 





























Normal Probability Plot 

Pretraining Teamwork Behavior Data 


Probability 

o . b 

OOON> cn 00 CD CD CD 

■^-^010 O O Ol CD CO 

_1_1_ 1 _ i _ 1 _l_ 1 _ 1 _1_ 
















• 













1* 




• 

• 




• 















1 1 1 r 

2 3 4 5 

Pretmwk 

Average: 3.83333 Anderson-Darling Normality Test 

Std Dev: 1.04994 A-Squared: 0.411 

Not data: 24 p-value: 0.316 































Normal Probability Plot 

Posttraining Teamwork Behavior Data 



Average: 4.40611 
Std Dev: 1.10595 
N of data: 24 


Posttmwk 


Anderson-Darling Normality Test 
A-Squared: 0.549 
p-value: 0.141 
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Normal Probability Plot 

Teamwork Behavior Gain Data 
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Average: 0.572778 
Std Dev: 1.03212 
N of data: 24 


Tmwkgain 


Anderson-Darling Normality Test 
A-Squared: 0.643 
p-value: 0.082 
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